Skip to content

Fix: Kill the server with one keyboard interrupt#94

Merged
lstein merged 3 commits intolstein/feature/nicer-shutdownfrom
copilot/enhancement-kill-server-with-one-interrupt
Feb 27, 2026
Merged

Fix: Kill the server with one keyboard interrupt#94
lstein merged 3 commits intolstein/feature/nicer-shutdownfrom
copilot/enhancement-kill-server-with-one-interrupt

Conversation

Copy link

Copilot AI commented Feb 27, 2026

Summary

Pressing Ctrl+C once printed a uvicorn traceback and required a second Ctrl+C to fully exit. There were two separate causes:

  1. Traceback on first Ctrl+C: uvicorn's capture_signals() re-raises the captured signal via signal.raise_signal() after graceful shutdown, causing an unhandled KeyboardInterrupt to bubble up through loop.run_until_complete(server.serve()).

  2. Process hanging after first Ctrl+C: Even after catching the KeyboardInterrupt, the process would hang indefinitely because Python's interpreter tried to join non-daemon background threads (e.g. the model install service thread) during shutdown.

Fix: wrap the serve call in run_app.py with a try/except KeyboardInterrupt that logs a clean shutdown message and then calls os._exit(0) to force an immediate process exit, bypassing Python's thread-joining phase:

try:
    loop.run_until_complete(server.serve())
except KeyboardInterrupt:
    logger.info("InvokeAI shutting down...")
    # Force exit to avoid hanging on non-daemon background threads (e.g. model install service).
    os._exit(0)

Uvicorn's graceful shutdown still runs normally before the KeyboardInterrupt is re-raised. Normal server exits (e.g. via API shutdown endpoint) are unaffected.

Related Issues / Discussions

QA Instructions

Start invokeai-web and press Ctrl+C once. The server should exit immediately and cleanly with an InvokeAI shutting down... log line, no traceback, and no need to press Ctrl+C a second time.

Merge Plan

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Tests added / updated (if applicable)
  • ❗Changes to a redux slice have a corresponding migration
  • Documentation added / updated (if applicable)
  • Updated What's New copy (if doing a release after this PR)
Original prompt

This section details on the original issue you should resolve

<issue_title>[enhancement]: Kill the server with one keyboard interrupt</issue_title>
<issue_description>### Is there an existing issue for this?

  • I have searched the existing issues

Contact Details

No response

What should this feature add?

Currently to kill invokeai-web from the command line, you must press ^C twice. The first ^C gives this:

^C[2026-02-26 18:56:13,606]::[ModelInstallService]::INFO --> Installer thread 139253501650624 exiting                                                                                                         
Traceback (most recent call last):                                                                      
  File "/home/lstein/invokeai-lstein/.venv/bin/invokeai-web", line 12, in <module>                                                                                                                              
    sys.exit(run_app())                                                                                 
             ^^^^^^^^^                                                                                  
  File "/home/lstein/Projects/InvokeAI-lstein/invokeai/app/run_app.py", line 103, in run_app                                                                                                                    
    loop.run_until_complete(server.serve())                                                                                                                                                                     
  File "/home/lstein/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/asyncio/base_events.py", line 678, in run_until_complete                                                            
    self.run_forever()                                                                                                                                                                                          
  File "/home/lstein/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/asyncio/base_events.py", line 645, in run_forever                                                                   
    self._run_once()                                                                                                                                                                                            
  File "/home/lstein/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/asyncio/base_events.py", line 1999, in _run_once                                                                    
    handle._run()                                                                                                                                                                                               
  File "/home/lstein/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/asyncio/events.py", line 88, in _run                                                                                
    self._context.run(self._callback, *self._args)                                                                                                                                                              
  File "/home/lstein/invokeai-lstein/.venv/lib/python3.12/site-packages/uvicorn/server.py", line 70, in serve                                                                                                   
    with self.capture_signals():                                                                                                                                                                                
         ^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                                 
  File "/home/lstein/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/contextlib.py", line 144, in __exit__                                                                               
    next(self.gen)                                                                                                                                                                                              
  File "/home/lstein/invokeai-lstein/.venv/lib/python3.12/site-packages/uvicorn/server.py", line 331, in capture_signals                                                                                        
    signal.raise_signal(captured_signal)   
KeyboardInterrupt                                                                                                                                                                      

The second ^C gives these messages and it does finally shut down:


</details>



<!-- START COPILOT CODING AGENT SUFFIX -->

- Fixes lstein/InvokeAI#93

<!-- START COPILOT CODING AGENT TIPS -->
---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our [2 minute survey](https://gh.io/copilot-coding-agent-survey).

Co-authored-by: lstein <111189+lstein@users.noreply.github.com>
Copilot AI changed the title [WIP] Enhance server termination with single keyboard interrupt Fix: Kill the server with one keyboard interrupt Feb 27, 2026
Copy link
Owner

@lstein lstein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Almost, but not quite. After running for a while I still have to press ^C twice. Here's the first time:

^C[2026-02-26 19:17:48,392]::[ModelInstallService]::INFO --> Installer thread 140553368897216 exiting
[2026-02-26 19:17:49,395]::[InvokeAI]::INFO --> InvokeAI shutting down...

I waited for about 30s and then ^C again:

^CException ignored in: <module 'threading' from '/home/lstein/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/threading.py'>
Traceback (most recent call last):
  File "/home/lstein/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/threading.py", line 1624, in _shutdown
    lock.acquire()
KeyboardInterrupt: 

… threads

Co-authored-by: lstein <111189+lstein@users.noreply.github.com>
Copy link
Author

Copilot AI commented Feb 27, 2026

Almost, but not quite. After running for a while I still have to press ^C twice. Here's the first time:

^C[2026-02-26 19:17:48,392]::[ModelInstallService]::INFO --> Installer thread 14055336889721...

The root cause was that after catching KeyboardInterrupt, the process still hung waiting for non-daemon background threads (model install service, etc.) to finish — hence the second Ctrl+C.

Fixed in commit 1f0f476 by calling os._exit(0) immediately after logging the shutdown message. This forces an immediate process exit, bypassing Python's thread-joining phase entirely. It's scoped only to the KeyboardInterrupt path, so a normal server stop still returns cleanly.

Copy link
Owner

@lstein lstein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works as advertised.

@lstein lstein marked this pull request as ready for review February 27, 2026 00:28
@lstein lstein merged commit eeff99f into lstein/feature/nicer-shutdown Feb 27, 2026
13 of 19 checks passed
@lstein lstein deleted the copilot/enhancement-kill-server-with-one-interrupt branch February 27, 2026 00:30
@lstein lstein restored the copilot/enhancement-kill-server-with-one-interrupt branch February 28, 2026 15:09
@lstein lstein deleted the copilot/enhancement-kill-server-with-one-interrupt branch February 28, 2026 15:10
lstein added a commit that referenced this pull request Mar 2, 2026
* Initial plan

* Handle KeyboardInterrupt in run_app to allow single Ctrl+C shutdown

Co-authored-by: lstein <111189+lstein@users.noreply.github.com>

* Force os._exit(0) on KeyboardInterrupt to avoid hanging on background threads

Co-authored-by: lstein <111189+lstein@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: lstein <111189+lstein@users.noreply.github.com>
lstein added a commit that referenced this pull request Mar 2, 2026
* Initial plan

* Handle KeyboardInterrupt in run_app to allow single Ctrl+C shutdown

Co-authored-by: lstein <111189+lstein@users.noreply.github.com>

* Force os._exit(0) on KeyboardInterrupt to avoid hanging on background threads

Co-authored-by: lstein <111189+lstein@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: lstein <111189+lstein@users.noreply.github.com>

Fix graceful shutdown to wait for download/install worker threads (#102)

* Initial plan

* Replace os._exit(0) with ApiDependencies.shutdown() on KeyboardInterrupt

Instead of immediately force-exiting the process on CTRL+C, call
ApiDependencies.shutdown() to gracefully stop the download and install
manager services, allowing active work to complete or cancel cleanly
before the process exits.

Co-authored-by: lstein <111189+lstein@users.noreply.github.com>

* Make stop() idempotent in download and model install services

When CTRL+C is pressed, uvicorn's graceful shutdown triggers the FastAPI
lifespan which calls ApiDependencies.shutdown(), then a KeyboardInterrupt
propagates from run_until_complete() hitting the except block which tries
to call ApiDependencies.shutdown() a second time.

Change both stop() methods to return silently (instead of raising) when
the service is not running. This handles:
- Double-shutdown: lifespan already stopped the services
- Early interrupt: services were never fully started

Co-authored-by: lstein <111189+lstein@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: lstein <111189+lstein@users.noreply.github.com>

Fix shutdown hang on session processor thread lock (#108)

* Initial plan

* Fix shutdown hang: wake session processor thread on stop() and mark daemon

Co-authored-by: lstein <111189+lstein@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: lstein <111189+lstein@users.noreply.github.com>
lstein added a commit that referenced this pull request Mar 6, 2026
…i#8936)

* Fix: Kill the server with one keyboard interrupt (#94)

* Initial plan

* Handle KeyboardInterrupt in run_app to allow single Ctrl+C shutdown

Co-authored-by: lstein <111189+lstein@users.noreply.github.com>

* Force os._exit(0) on KeyboardInterrupt to avoid hanging on background threads

Co-authored-by: lstein <111189+lstein@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: lstein <111189+lstein@users.noreply.github.com>

Fix graceful shutdown to wait for download/install worker threads (#102)

* Initial plan

* Replace os._exit(0) with ApiDependencies.shutdown() on KeyboardInterrupt

Instead of immediately force-exiting the process on CTRL+C, call
ApiDependencies.shutdown() to gracefully stop the download and install
manager services, allowing active work to complete or cancel cleanly
before the process exits.

Co-authored-by: lstein <111189+lstein@users.noreply.github.com>

* Make stop() idempotent in download and model install services

When CTRL+C is pressed, uvicorn's graceful shutdown triggers the FastAPI
lifespan which calls ApiDependencies.shutdown(), then a KeyboardInterrupt
propagates from run_until_complete() hitting the except block which tries
to call ApiDependencies.shutdown() a second time.

Change both stop() methods to return silently (instead of raising) when
the service is not running. This handles:
- Double-shutdown: lifespan already stopped the services
- Early interrupt: services were never fully started

Co-authored-by: lstein <111189+lstein@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: lstein <111189+lstein@users.noreply.github.com>

Fix shutdown hang on session processor thread lock (#108)

* Initial plan

* Fix shutdown hang: wake session processor thread on stop() and mark daemon

Co-authored-by: lstein <111189+lstein@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: lstein <111189+lstein@users.noreply.github.com>

* Fix: shut down asyncio executor on KeyboardInterrupt to prevent post-generation hang (#112)

Fix: cancel pending asyncio tasks before loop.close() to suppress destroyed-task warnings
Fix: suppress stack trace when dispatching events after event loop is closed on shutdown
Fix: cancel in-progress generation on stop() to prevent core dump during mid-flight Ctrl+C

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: lstein <111189+lstein@users.noreply.github.com>

---------

Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: lstein <111189+lstein@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants